The present invention relates to the processing performed for a recovered image during a digital image compression process, and relates in particular to a deblocking filtering apparatus and method defined in MPEG-4.
For the processing of digital images, the internationally accepted MPEG digital image compression standards provide for the employment of a discrete cosine transform. This is an irreversible transform used for quantization, and depending on the condition of an original image, a phenomenon occurs whereby a pseudo outline having a block shape (block noise), which is inherent to the system, may appear in a recovered image.
As a countermeasure, Vertification Model 7.0 of the MPEG-4 video standards defines a deblocking filter as a post process for a recovered image. As is shown in FIG. 3, the defined deblocking filter performs filtering for ten received pixel values positioned around the block boundary, and outputs eight pixel values positioned around the block boundary. Two types of operation modes, a DC offset mode operation (hereinafter referred to as a D mode operation) and a default mode operation, are switched in accordance with a change value (hereinafter referred to as an activity) for the value of a pixel near the block boundary. An evaluation function representing the activity employs the following equation (1).f=φ{g(0)−g(1)}+φ{g(1)−g(2)}+φ{g(2)−g(3)}+φ{g93)−g(4)}+φ{g(4)−g(5)}+φ{g(5)−g(6)−g(7)}+φ{g(7)−g(8)}+φ{g(8)−g(9)}whereif (abs(x)<=Th1) φ(x)=1;else φ(x)=0  (1)
According to an evaluation function f in equation (1), for ten pixel values positioned around a block boundary, an absolute differential value for the difference between adjacent pixel values is compared with a threshold value Th1, and a count is acquired of the locations whereat the absolute differential value is equal to or smaller than the threshold value Th1. If the value of the evaluation function f is equal to or greater than a threshold value Th2, i.e., when the activity is low, the D mode operation is selected. But if the value of the evaluation function f is smaller than the threshold value Th2, i.e., when the activity is high, the default mode operation is selected.
A more effective smoothing process is performed for a D mode operation than is performed for a default mode operation, as is illustrated by an operation equation (2) that is shown below. The smoothing process is performed if the absolute differential value between a maximum pixel value and a minimum pixel value for a string of eight pixels positioned around a block boundary is smaller than twice a quantization parameter QP. If the absolute differential value is equal to or greater than twice the quantization parameter QP, the smoothing process is not performed and an input pixel value is output unchanged.MAX=max{(g(1), g(2), g(3), g(4), g(5), g(6), g(7), g(8)}; MIN=min{(g(1), g(2), g(3), g(4), g(5), g(6), g(7), g(8)}; if(abs(MAX−MIN))<2*QP){min#padding=abs(g(0)−g(1))<QP?g(0):g(1); max#padding=abs(g(8)−g(9))<QP?g(9):g(8); g′(n)=0; for(i=−4;i<5;i++) g′(n)+=coef(i+4)*(n+1<1?min#padding:n+i>8?max#padding:g(n+i))); . . . (a) g′(n)=(g′(n)+8)>>4; }else g′(n)=g(n) (where coef(9)={1,1,2,2,4,2,2,1,1})  (2)
An equation (3) is obtained by specially developing formula (a) in equation (2), and the smoothing process is performed by a weighting function employed mainly for a target pixel and the logical sum of the input pixel string.g′(1)=min+min+2*min+2*min+4*g(1)+2*g(2)+2*g(3)+g(4)+g(5); g′(2)=min+min+2*min+2*g(1)+4*g(2)+2*g(3)+2*g(4)+g(5)+g(6); g′(3)=min+min+2*g(1)+2*g(2)+4*g(3)+2*g(4)+2*g(5)+g(6)+g(7); g′(4)=min+g(1)+2*g(2)+2*g(3)+4*g(4)+2*g(5)+2*g(6)+g(7)+g(8); g′(5)=g(1)+g(2)+2*g(3)+2*g(4)+4*g(5)+2*g(6)+2g(7)+g(8)+max; g′(6)=g(2)+g(3)+2*g(4)+2*g(5)+4*g(6)+2*g(7)+2*g(8) +max+max; g′(7)=g(3)+g(4)+2*g(5)+2*g(6)+4*g(7)+2*g(8)+2+max+max+max; g′(8)=g(4)+g(5)+2*g(6)+2*g(7)+4*g(8)+2+max+2max+max+; min:min#padding, max:max#padding  (3)
An equation (4) is employed for the default mode operation. The smoothing process is performed only for two pixels along a block boundary, and the input values for the other pixels are output unchanged.max=(g(4)−g(5))/2d=nint(5*(act0′−act0)/8); if((abs(act0)<QP)&&(sign(max)==sign(d))) d=(abs(d)>abs(max))?max:d; else d=0; g′(4)=g(4)−d; g′(5)=g(5)−d; where QP is a quantization parameter of a macro block to which g(5) belongs, act0=nint((2*g(3)−5*g(4)+5*g(5)−2*g(6))/8); act1=nint((2*g(1)−5*g(2)+5*g(3)−2*g(4))/8); act2=nint((2*g(5)−5*g(6)+5*g(7)−2*g(8))/8); act0′=sign(act0)·min(abs(act0), abs(act1), abs(act2)), where no operation is performed for g(1) to g(3) and g(6) to g(8)  (4)
Further, instead of the default mode operation, Telenor's adaptive filtering (hereinafter referred to as a T mode operation), which has a smaller calculation amount, may be employed, as is illustrated by a T mode operation (5) that follows. As well as in the default mode operation, the smoothing process is performed for only two pixels along a block boundary, and the input values of all other pixels are output unchanged. For the two pixels along the block boundary, when the evaluation function d in equation (5) is equal to or smaller than a quantization parameter QP/2, a smoothing process is performed in which the value of the evaluation function d is added to or is subtracted from the input pixel value. Then, if the evaluation function value d is greater than the quantization parameter QP/2, the input pixel value is output unchanged.d=(g(3)−3*g(4)+3*g(5)−g(6)+4)>>3; g′(4)=abs(d)>(QP/2)?g(4):g(4)+d; g′(5)=abs(d)>(QP/2)?g(5):g(5)−d, where no operation is performed for g(1) to g(3) and g(6) to g(8)  (5)
The above deblocking filtering defined by MPEG-4 is first performed along all horizontal edges, and is then performed along all vertical edges.
The above deblocking filter operation is performed, using software, by a general-purpose calculator in a processor, or part of the deblocking filter operation is performed by special hardware and the remainder of the operation is performed, using software, by a general-purpose calculator in a processor.
FIG. 4 is a block diagram showing the special hardware operation for formula (a) in the D mode operation equation (2). In FIG. 4, a counter 100 counts processing cycles from 0 to 7, operation blocks 101 to 108 correspond to filtering target pixels n (n is an integer from 1 to 8), and an output selector 109 selects one of the outputs of the operation blocks 101 to 108.
The operation block 101 includes: first and second selectors n11 and n12, for receiving at least two of the externally input data max_padding, min_padding, input pixel values g(x) and g(x+1), and data 0; a third selector n13, for receiving data “8” and the output of a register n17, which will be described later; first and second shifters n14 and n15, for shifting the output of both the first and the second selectors n11 and n12; an adder n16 for adding the output of the first and the second shifters n14 and n15 to the output of the third selector n13; the register n17, for holding the output of the adder n16; and a third shifter n18, for shifting the output of the register n17.
The operation blocks 102 to 108 have the same configuration as the operation block 101, and include, respectively, first selectors n21, n31, n41, n51, n61, n71 and n81; seconds electors n22, n32, n42, n52, n62, n72 and n82; third selectors n23, n33, n43, n53, n63, n73 and n83; first shifters n24, n34, n44, n54, n64, n74 and n84; second shifters n25, n35, n45, n55, n65, n75 and n85; adders n26, n36, n46, n56, n66, n76 and n86; registers n27, n37, n47, n57, n67, n77 and n87; and third shifters n28, n38, n48, n58, n68, n78 and n88.
Since software is used for conventional deblocking filtering, a large number of operating cycles is required and the processing is slow. Further, when as in the conventional art special hardware is used for a part of an operation in order to increase the processing speed, both a processor and special hardware are required, and this increases manufacturing costs.